Source of Acquisition 
NASA Johnson Space Center 


Human Centered Autonomous and Assistant Systems Testbed for Exploration Operations 

Jane T. Malin, Frances Mount, Patricia Carreon and Susan E. Tomey 

The Engineering and Mission Operations Directorates at NASA Johnson Space Center are 
combining laboratories and expertise to establish the Human Centered Autonomous and 
Assistant Systems Testbed for Exploration Operations. This is a testbed for human centered 
design, development and evaluation of intelligent autonomous and assistant systems that will be 
needed for human exploration and development of space. This project will improve human- 
centered analysis, design and evaluation methods for developing intelligent software. This 
software will support human-machine cognitive and collaborative activities in future 
interplanetary work environments where distributed computer and human agents cooperate. We 
are developing and evaluating prototype intelligent systems for distributed multi-agent mixed- 
initiative operations. The primary target domain is control of life support systems in a planetary 
base. Technical approaches will be evaluated for use during extended manned tests in the target 
domain, the Bioregenerative Advanced Life Support Systems Test Complex (BlO-Plex). A spin- 
off target domain is the International Space Station (ISS) Mission Control Center (MCC). 
Products of this project include human-centered intelligent software technology, innovative 
human interface designs, and human-centered software development processes, methods and 
products. The testbed uses adjustable autonomy software and life support systems simulation 
models from the Adjustable Autonomy Testbed, to represent operations on the remote planet. 
Ground operations prototypes and concepts will be evaluated in the Exploration Planning and 
Operations Center (ExPOC) and Jupiter Facility. 
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Overview 


HCAAST goals and challenges 
Human Centered analysis and design work 
Intelligent systems research challenges 

- Mixed initiative interaction with distributed 
intelligent agents 

- Data understanding 

- User interfaces 

HCAAST phasing and FY01 plan 


HCAAST Goals 


Develop, integrate and validate human 
centered intelligent systems technology 

- Mission operations support systems with 
embedded intelligent systems technology 

- Human centered analysis and design for 
intelligent systems 

- Intelligent systems technology components 


A Joint Engineering and 
Operations Testbed 

Management team and networked labs from 
Engineering and Mission Operations Directorates 

- Exploration-class mission control communications, data 
storage and workstations 

- Intelligent Systems Laboratory (Engineering) 

• Intelligent autonomous control of simulated production plant on 
Mars 

- Quest Lab, RaPID Lab and ExPOC (Mission Operations) 

• Intelligent assistant systems on earth 

- BioPLEX target (Engineering) 

• Initial focus on air processing, transfer and storage for life support 



Cooperating Intelligent Agents 


Adjustable Autonomous Agents 

- Plan/schedule, execute, monitor, safe and recover 

- Detect, diagnose, assess 

- Recommend, predict, evaluate plans and actions 

- Adjust level of autonomy and support mixed initiative 

Intelligent Briefing and Response Assistant Agents 

- Monitor, make console log entries and alert 

- Analyze data patterns 

- Search 

- Build and organize briefings 


Challenges for Autonomous Agents 


Autonomous remote processing plants with 
limited communication 

- Delays of up to one hour and low bandwidth 

Shifting between autonomy and teamwork to 
handle inevitable unexpected problems 

Loose coordination for flexible teams 

- Mixed initiative interaction and coordination 

- Updates and orientation rather than monitoring 

- Loose commanding rather than synchronized traded 
control 


Leveraging Projects for 
Component Technology 

• Adjustable Autonomy Testbed/Cooperative Fault 
Management (CETDP projects) 

- 3T/Livingstone agent architecture and user interfaces 

- CONFIG hybrid models and simulation of operation of 
planetary systems hardware 

• Operations Assistants Technology (MOD) 

- Electronic Console Logging prototype 

• Anomaly Response Analysis Archiving (previous 
Code U NRA and MOD) 

- ARTIS anomaly response team archive prototype 

• NRAs and other coordinated in-house projects 


8 


Cooperative Adjustable Autonomous 
Fault Management Projects 

• HCAAST leverages CETDP projects with simulated 
autonomous operations on the planet 

- Scenario-based hybrid simulation of life support hardware 
and operations - initially gas processing 

- Adjustable autonomous control and fault management by 
3T and Livingstone agents 

• Planner with goals and plans 

• Reactive activity layer with tasks and intentions 

• Model-based agent with state assessments, diagnoses and 
recommended recovery actions 

- Focus of these projects in FY01 is on cooperative fault 
management - situation assessment and conflict resolution 


Model of Planetary Life Support System Testbed 
used to Evaluate Intelligent Control Software 



02 -FC MV-O2-105 
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Human Centered Technology for 

Analysis and Design 

Make intelligent systems team players 

- Study exploration mission operations team analogs, to 
design cooperating intelligent systems 

- Design intelligent agents for mixed initiative interaction 

- Evaluate intelligent agents in mixed initiative interaction 
scenarios 

Make analysis products more useful and usable for 

developers of intelligent systems technology 

- Models and other products from field studies and 
evaluations 

- Investigate just-in-time analysis for levels of design 


Three Levels of Analysis for 

System Design 

• Work system objectives, strategies, operations 
concepts 

- Work flow models, artifact models 

• Functions, tasks, activities, work and system 
structures 

- Structured models of supported tasks, activities 

• Information, displays and controls 

- Decisions to support, information, communication 
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Support for Intelligent Systems 
Technology Development 

• TRL 4 Laboratory validation of in-house and 
externally developed technology 

- Prepare for TRL-5/6 exploration testbeds 

• BIO-Plex life support testbed 

• Mission Control testbeds in Quest Lab and ExPOC 

• TRL 3 Proof-of-concept studies 

- Support with system-level simulation of operations scenarios 

• TRL 1 -2 Challenges for exploration operations 

- Automated Reasoning: Mixed initiative interaction with 
distributed intelligent agents 

- Intelligent Data Understanding 

- Design for Future User interfaces 
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Roles of Autonomous Systems 

When situation is routine, autonomous system is 
like flight controller, loosely coordinating with 
rest of team - periodic update reports 

When recognizing a problem, autonomous system 
is like Station Duty Officer 

- Notify, analyze, report with assessments, near-term 
actions, predictions 

When solving a problem, autonomous system is 
like member of anomaly response team 

- Continue to function as remote “flight controller” 

- Compare and resolve conflicts, evaluate, update and 
rehearse chosen course of action 


Mixed Initiative Interaction 
Research Challenges 

• Agents on planet and on earth, with variable 
communication delays 

• Provide information to establish shared frame of 
reference in periodic updating of on-call operators 

- Communicate high level assessments, plans, operations 

• Communicate among heterogeneous agents 
managing multiple complex systems 

• Give and get help with anomaly response 

• Handle loose commanding and delegation 

• Negotiate and evaluate alternatives 
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Results of Human-centered Research 


• Mission Control Center: a place where teams 
manage unexpected situations 

- Loose coordination via voice loops, console logs or 
diaries, shift change updates 

- Anomaly response teams with diverse viewpoints 

• HC Research: Analyses, models, designs for group 
communication and information processing support 

- Common ground and common representations supporting 
awareness for coordination 

- Conflict resolution that leads to better team assessments, 
goals and plans 
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Common Ground 


• Updates replace real-time monitoring 

- Briefings and logs rather than conversation 

- High level, with access to lower-level information and 
data 

- Information above data level, organized for review and 
revision 

• Plans, situations, issues and recommendations 

• Goal is to relieve some escalation pressures by 
establishing common ground of awareness and 
providing appropriate level of information 
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Conflict Resolution Challenges 

Long-distance “negotiation” and delegation for 
anomaly response 

Expose alternatives and potential conflicting 
assessments, interests/goals, operations 

Support generation of options and analysis and 
evaluation of assessments and plans 

- “Current” and predicted status and capabilities 
communicated to ground 

Design for loose commanding (delegation) in face 
of uncertain state on planet after time delay 

- Goals and conditions communicated to planet with 
acknowledgements and commands 


Challenges of Changing Plans 

and Procedures 

• Communicating novel procedures and plans 

- Use activity trees to change operations without 
programming? 

• Assessing feasibility, safety and mission 
impacts of new procedures and plans 

- On earth and on the planet 

• How are procedure and plan updates 
handled between ground and space now? 
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Research Challenges for Data 
Understanding & Human Interfaces 

• Operate remote complex systems autonomously 
with limited communication 

- Interpret, filter, correlate, summarize and manage 

• Planetary systems data 

• High level autonomy information 

- Adapt and learn 

• Design for integration of novel interfaces for 
exploration operations 

- Adaptive interfaces 

- Multimodal interfaces (e.g., interactions with EVA 
robotic assistant, virtual presence) 
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HCAAST Phasing Approach 

• FY01-03 Cooperative autonomous planetary 
operations and fault management of life support 
systems, with mission controllers on earth 

- Distributed intelligent briefing and response assistants for 
orientation and intervention 

• FY03-05 Cooperative autonomous planetary 
operations and fault management of life support 
systems, with earth and planet (crew) mission 
controllers 

• FY04-05 Given appropriate funding, multi-modal 
user interfaces for distributed mobile agents 
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HCAAST FY01 Plan 


• Establish testbed of networked labs, with 
equipment to support analysis and evaluation 

• Select, use and evaluate usefulness of HC 
analysis, design and evaluation methods and 
products, for HCAAST system developers 

• Design, prototype and evaluate initial HCAAST 
BlO-Plex oriented systems 

• Support spin-offs to JSC Mission Operations 
Directorate Advanced Development groups 
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